*Replication Instructions for

*Countering International Anti-LGBT+ Backlash by Discrediting the Messengers: Evidence from Survey Experiments in Bosnia

*Phillip Ayoub, Douglas Page, Sam Whitt

*Below are instructions for replicating all manuscript and online appendix tables and figures in STATA using the dataset "PRQ messenger replication data.dta". Please contact Sam Whitt (swhitt@highpoint.edu) for questions regarding data replication. See also the dofile "PRQ messenger replication do file". 

*Note: You may need to install STATA packages for the catcibar, cibar, and iebaltab commands. Use findit with the command name to identify and download the appropriate packets to install. 

*Note: In addition, some graphs require additional formatting using filename.grec files with the graph play command. To format a graph, simply run the command to generate the graph in the do file in STATA, then open the "Graph Editor" in STATA and click on the GREEN "Play Recording" button, then select "Browse" to select the grec file from the folder "grec files for STATA graph formatting" among Replication files. The name of the grec file is indicated in the note below the graph command in the do file for the specific graph you wish to format. This should automatically format the graph, which you may then save to a location of your choosing.

*Manuscript Replication

*"Stata user generated commands to install for replication purposes"

*"cibar"

ssc install cibar, replace

*"catcibar"

net install catcibar, from("https://aarondwolf.github.io/catcibar")

*"iebaltab from ietoolkit"

ssc install ietoolkit, replace

*In Text Replication

*Serbs who receive the Putin anti-LGBT+ messaging treatment are more likely to support a political party favoring equal rights for LGBT+ people (model 1, Cohen's D=0.22) and marriage equality (model 2, Cohen's D=0.22).

esize twosample equalrights if  serb==1 & experiment1~=2, by(experiment1)
esize twosample gaymarriage if  serb==1 & experiment1~=2, by(experiment1)

*Among Serbs, 80% strongly agree with the statement in the control group compared to 65-69% in the treatment groups  (only 48% of non-Serbs strongly agree on average). 

tab western if serb==1 & experiment1==3
tab western if serb==1 & experiment1==2
tab western if serb==1 & experiment1==1

tab western if serb==0 
tab western if serb==0 & experiment1==3 
tab western if serb==0 & experiment1==2 
tab western if serb==0 & experiment1==1

*More importantly, linking Dodik to Putin undermines support for the anti-Western contestation that "homosexuality is not part of Bosnian culture"  (model 4)– a statement that 94% of Serbs strongly agree with in the control group but is reduced to 75-70% in the treatment groups.

tab notinculture if serb==1 & experiment2==4
tab notinculture if serb==1 & experiment2==3
tab notinculture if serb==1 & experiment2==2
tab notinculture if serb==1 & experiment2==1

*Table 3

sum bosniajul i.ethnicity women age educationb unemployedb ruralb 

*Figure 1

catcibar equalrights gaymarriage protectgay dodikfav putinsupport russianvictory if  ethnicity<4, over(ethnicity)
*Note additional formatting requires the "Figure 1 Formatting.grec" file with the command graph play "Figure 1 Formatting.grec" 

*Table 4

reg equalrights ib3.experiment1##serb , robust
reg gaymarriage ib3.experiment1##serb  , robust
reg putinsupport ib3.experiment1##serb  , robust
reg western ib3.experiment1##serb , robust

*Figure 2

cibar equalrights if  serb==1, over1(exp1ct)
graph save g1, replace
cibar gaymarriage if  serb==1, over1(exp1ct)
graph save g2, replace
cibar putinsupport if  serb==1, over1(exp1ct)
graph save g3, replace
cibar western if  serb==1, over1(exp1ct)
graph save g4, replace
graph combine "g1.gph" "g2.gph" "g3.gph" "g4.gph" 
*Note additional formatting requires the "Figure 2 Formatting.grec" file with the command graph play "Figure 2 Formatting.grec" 

*Table 5

reg protectgay ib4.experiment2##serb , robust
reg gaymarriage2 ib4.experiment2##serb , robust
reg dodikfav ib4.experiment2##serb , robust
reg western ib4.experiment2##serb , robust
reg notinculture ib4.experiment2##serb , robust

*Figure 3

cibar protectgay if  serb==1, over1(exp2ct)
graph save g5, replace
cibar gaymarriage2 if  serb==1, over1(exp2ct)
graph save g6, replace
cibar dodikfav if  serb==0, over1(exp2ct)
graph save g7, replace
cibar notinculture if  serb==1, over1(exp2ct)
graph save g8, replace
graph combine "g5.gph" "g6.gph" "g7.gph" "g8.gph" 
*Note additional formatting requires the "Figure 3 Formatting.grec" file with the command graph play "Figure 3 Formatting.grec" 

*Table 6

reg protectgay  serb bosniak women age educationb unemployedb ruralb revimpreligion revimpethnicity russiafav ukraineunfav putinsupport russianvictory western notinculture , robust

*Online Appendix Replication

*Summary of Demographics (July + October 2023)

sum i.ethnicity women age educationb unemployedb ruralb rs 

*Summary of Demographics (July 2023)

sum i.ethnicity women age educationb unemployedb ruralb rs if bosniajul==1

*Summary of Demographics (October 2023)

sum i.ethnicity women age educationb unemployedb ruralb rs if bosniaoct==1

*Summary Table of All Variables in Analysis

sum equalrights gaymarriage putinsupport western 
sum protectgay gaymarriage2 dodikfav western 

sum i.ethnicity women age educationb unemployedb ruralb rs 

sum revimpreligion revimpethnicity russiafav ukraineunfav putinsupport russianvictory western notinculture 

*Balance and Matching on Covariates (Experiment 1, July+October)

iebaltab ethnicity women age educationb unemployedb ruralb rs , groupvar(experiment1) savexlsx(balance1)

*Balance Tests (July 2023)

iebaltab ethnicity women age educationb unemployedb ruralb rs if bosniajul==1, groupvar(experiment1) savexlsx(balance1a)

*Balance Tests (October 2023)

iebaltab ethnicity women age educationb unemployedb ruralb rs if bosniaoct==1, groupvar(experiment1) savexlsx(balance1b)

*Balance and Matching on Covariates (Experiment 2, July+October)

iebaltab ethnicity women age educationb unemployedb ruralb rs , groupvar(experiment2) savexlsx(balance2)

*Balance Tests (July 2023)

iebaltab ethnicity women age educationb unemployedb ruralb rs if bosniajul==1, groupvar(experiment2) savexlsx(balance2a)

*Balance Tests (October 2023)

iebaltab ethnicity women age educationb unemployedb ruralb rs if bosniajul==1, groupvar(experiment2) savexlsx(balance2b)

*Experiment 1 robustness checks

*Average Treatment Effects from Experiment 1 (OLS Regression)

reg equalrights ib3.experiment1##serb , robust
reg gaymarriage ib3.experiment1##serb  , robust
reg putinsupport ib3.experiment1##serb  , robust
reg western ib3.experiment1##serb , robust

*Average Treatment Effects from Experiment 1 (OLS Regression, extended controls)

reg equalrights ib3.experiment1##serb women age educationb unemployedb ruralb bosniajul  , robust
reg gaymarriage ib3.experiment1##serb women age educationb unemployedb ruralb bosniajul   , robust
reg putinsupport ib3.experiment1##serb women age educationb unemployedb ruralb bosniajul   , robust
reg western ib3.experiment1##serb women age educationb unemployedb ruralb bosniajul   , robust

*Experiment 2 robustness checks

*Average Treatment Effects from Experiment 2 (OLS Regression)

reg protectgay ib4.experiment2##serb , robust
reg gaymarriage2 ib4.experiment2##serb , robust
reg dodikfav ib4.experiment2##serb , robust
reg western ib4.experiment2##serb , robust
reg notinculture ib4.experiment2##serb , robust

*Average Treatment Effects from Experiment 2 (OLS Regression, Extended Controls)

reg protectgay ib4.experiment2##serb women age educationb unemployedb ruralb bosniajul , robust
reg gaymarriage2 ib4.experiment2##serb women age educationb unemployedb ruralb bosniajul , robust
reg dodikfav ib4.experiment2##serb women age educationb unemployedb ruralb bosniajul , robust
reg western ib4.experiment2##serb women age educationb unemployedb ruralb bosniajul , robust
reg notinculture ib4.experiment2##serb women age educationb unemployedb ruralb bosniajul , robust


*Power Calculations and Effect Size Estimation (Experiment 1)

*Effect Size Estimations using One-War ANOVA (Sample Size)

power oneway, n(2000) ngroups(3) power(0.80 0.90 0.95 0.99)

*Effect Size Estimation using One-Way ANOVA (N per treatment group)

tab experiment1 
power oneway, ngroups(3) n1(862) n2(858) n3(307) power(0.80 0.90 0.95 0.99)

*Distribution of Respondents by Serb Moderator and Treatment Groups.

tab experiment1 serb
power oneway, ngroups(3) n1(360) n2(359) n3(307) power(0.80 0.90 0.95 0.99)

*Experiment 1 Effect Size for Select Treatment Effects (Cohen's D)
esize twosample equalrights if  serb==1 & experiment1~=2, by(experiment1)
esize twosample gaymarriage if  serb==1 & experiment1~=2, by(experiment1)
esize twosample putinsupport if  serb==1 & experiment1~=2, by(experiment1)
esize twosample western if  serb==1 & experiment1~=2, by(experiment1)

esize twosample putinsupport if  serb==1 & experiment1~=1, by(experiment1)
esize twosample western if  serb==1 & experiment1~=1, by(experiment1)

*Power Calculations and Effect Size Estimation (Experiment 2)

*Effect Size Estimations using One-War ANOVA (Sample Size)

power oneway, n(2000) ngroups(3) power(0.80 0.90 0.95 0.99)

*Effect Size Estimation using One-Way ANOVA (N per treatment group)

tab experiment2 
power oneway, ngroups(4) n1(574) n2(621) n3(553) n4(279) power(0.80 0.90 0.95 0.99)

*Distribution of Respondents by Serb Moderator and Treatment Groups.

tab experiment2 serb 
power oneway, ngroups(4) n1(574) n2(621) n3(553) n4(279) power(0.80 0.90 0.95 0.99)

*Experiment 2 Effect Size for Select Treatment Effects (Cohen's D)

esize twosample gaymarriage2 if  serb==1 & experiment2~=2 & experiment2~=3, by( experiment2)
esize twosample western if  serb==1 & experiment2~=2 & experiment2~=3, by(experiment2)

esize twosample gaymarriage2 if  serb==1 & experiment2~=1 & experiment2~=3, by(experiment2)
esize twosample western if  serb==1 & experiment2~=1 & experiment2~=3, by(experiment2)

esize twosample protectgay if  serb==1 & experiment2~=1 & experiment2~=2, by(experiment2)
esize twosample gaymarriage2 if  serb==1 & experiment2~=1 & experiment2~=2, by(experiment2)
esize twosample western if  serb==1 & experiment2~=1 & experiment2~=2, by(experiment2)

esize twosample dodikfav if  serb==0 & experiment2~=2 & experiment2~=3, by(experiment2)

esize twosample dodikfav if  serb==0 & experiment2~=1 & experiment2~=3, by(experiment2)
esize twosample western if  serb==0 & experiment2~=1 & experiment2~=3, by(experiment2)

esize twosample dodikfav if  serb==0 & experiment2~=1 & experiment2~=2, by(experiment2)
esize twosample western if  serb==0 & experiment2~=1 & experiment2~=2, by(experiment2)

log close